feat(over window): window function `RANGE` frame support in frontend #14648

stdrc · 2024-01-18T07:34:36Z

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This PR:

Adds RANGE frame support in parser and binder;
Adds necessary structs in call.rs to represent concepts related to range frame, including RangeFrameBounds and RangeFrameOffset.

15x out of 6xx lines of addition are planner tests, reviewers plz don't panic.

Checklist

I have written necessary rustdoc comments
I have added necessary unit tests and integration tests
I have added test labels as necessary. See details.
I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
All checks passed in ./risedev check (or alias, ./risedev c)
My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

stdrc · 2024-01-18T07:34:58Z

Current dependencies on/for this PR:

main
- PR fix(dashboard): fix file format to pass linting #14948
  - PR feat(over window): window function RANGE frame support in frontend #14648 👈
    - PR feat(over window): window function RANGE frame support - backend part #14416

This stack of pull requests is managed by Graphite.

fuyufjh

Generally LGTM

fuyufjh · 2024-02-02T05:19:06Z

src/frontend/src/binder/expr/function.rs

+        let mut offset = self.bind_expr(offset)?;
+        if !offset.is_const() {
+            return Err(ErrorCode::InvalidInputSyntax(
+                "offset in window frame bounds must be constant".to_string(),


We have a ConstEvalRewriter. Perhaps it can help here.

BTW, I saw we did constant folding at the last step in optimization.

risingwave/src/frontend/src/optimizer/mod.rs

Lines 399 to 400 in 8fdd0ba

// Const eval of exprs at the last minute

plan = const_eval_exprs(plan)?;

Shall we do it at the beginning instead? @kwannoel cc. @chenzl25

We can use ConstEvalRewriter to do constant folding if necessary. Here is the binder phase.

Oh, indeed. Here is the binder phase 😄

Yep, I do constant folding in binder phase here because I want to bind the expression to well-formed Rust types FrameBound<usize> and FrameBound<ScalarImpl>, so that later process can be easy. I guess it does few harm since it should always be simply a constant value/expr.

fuyufjh · 2024-02-02T05:37:18Z

proto/expr.proto

+  // The order type of order column in `RANGE` frame.
+  optional common.OrderType order_type = 15;
+  // The data type of offset value in `RANGE` frame.
+  optional data.DataType offset_data_type = 20;


It would look better if we split the the oneof offset as well. IIUC, the integer there was designed for row frames while datum was designed for range frames. The result will be like:

oneof Frame { RowFrame row = 1; RangeFrame range = 2; } message RowFrame {...} message RangeFrame {...}

However I also consider it's hard to keep compatibablity in this way...

You're right! I adjusted the proto messages in the latest commit 4f49c79 to deprecate the old fashion and align the messages with Rust types. Seems not that hard to maintain compatibility.

src/common/src/types/mod.rs

Signed-off-by: Richard Chien <[email protected]>

…nd` to `FrameBound<RangeFrameOffset>` Signed-off-by: Richard Chien <[email protected]>

This was referenced Jan 18, 2024

feat(common): add cmp_by to Sentinelled #14578

Merged

feat(common): derive Copy for DeltaBTreeMap #14579

Merged

github-actions bot added the type/feature label Jan 18, 2024

This was referenced Jan 18, 2024

refactor(over window): simplify find_affected_ranges and correct some corner cases #14580

Merged

refactor(over window): generalize WindowBuffer and window function AggregateState #14647

Merged

This was referenced Jan 18, 2024

feat(over window): window function RANGE frame support - backend part #14416

Merged

feat(over window): range frame support - frontend part #14094

Closed

stdrc force-pushed the rc/generalize-window-buffer branch from 27c8004 to 980d29d Compare January 18, 2024 09:55

stdrc force-pushed the rc/range-frame-frontend branch from 11fef80 to 7e0f0f8 Compare January 18, 2024 09:55

stdrc force-pushed the rc/generalize-window-buffer branch from 980d29d to dfe1a5b Compare January 19, 2024 05:24

stdrc force-pushed the rc/range-frame-frontend branch 2 times, most recently from 15a1f60 to 4398f57 Compare January 19, 2024 09:30

stdrc force-pushed the rc/generalize-window-buffer branch from c25c28f to 8f5ed91 Compare January 22, 2024 05:08

stdrc force-pushed the rc/range-frame-frontend branch from 49dd458 to 5fba65c Compare January 22, 2024 05:08

stdrc force-pushed the rc/generalize-window-buffer branch from 8f5ed91 to d3eba36 Compare January 23, 2024 07:27

stdrc force-pushed the rc/range-frame-frontend branch from 5fba65c to ea66a9c Compare January 23, 2024 07:27

stdrc force-pushed the rc/generalize-window-buffer branch from d3eba36 to 9933c72 Compare January 24, 2024 08:09

stdrc force-pushed the rc/range-frame-frontend branch from a4febe8 to d2172f3 Compare January 24, 2024 08:09

stdrc force-pushed the rc/generalize-window-buffer branch from 9933c72 to d23eeca Compare January 24, 2024 08:57

stdrc force-pushed the rc/range-frame-frontend branch from d2172f3 to 2264345 Compare January 24, 2024 08:57

stdrc marked this pull request as ready for review January 24, 2024 09:28

stdrc requested review from TennyZhuang, xiangjinwu, BugenZhao and st1page January 24, 2024 09:28

stdrc force-pushed the rc/generalize-window-buffer branch from d23eeca to bd2ed75 Compare January 24, 2024 10:06

stdrc requested a review from a team as a code owner January 24, 2024 10:06

stdrc force-pushed the rc/range-frame-frontend branch from f4e3e18 to 68b4623 Compare January 24, 2024 10:06

stdrc force-pushed the rc/generalize-window-buffer branch from bd2ed75 to 16f5d06 Compare January 25, 2024 04:07

stdrc force-pushed the rc/range-frame-frontend branch 3 times, most recently from 46f031e to 8566755 Compare February 2, 2024 05:20

stdrc changed the base branch from main to rc/fix-dashboard-format February 2, 2024 05:20

stdrc mentioned this pull request Feb 2, 2024

fix(dashboard): fix file format to pass linting #14948

Merged

9 tasks

fuyufjh approved these changes Feb 2, 2024

View reviewed changes

stdrc commented Feb 2, 2024

View reviewed changes

src/common/src/types/mod.rs Outdated Show resolved Hide resolved

Base automatically changed from rc/fix-dashboard-format to main February 4, 2024 04:53

stdrc added 16 commits February 4, 2024 16:58

fix dashboard check

129349a

Signed-off-by: Richard Chien <[email protected]>

add RangeFrameBounds and FrameBounds::Range variant

b8c4830

Signed-off-by: Richard Chien <[email protected]>

parse and bind range frames

6518c36

Signed-off-by: Richard Chien <[email protected]>

mark range frames as unimplemented when generating physical plan

1b304b9

Signed-off-by: Richard Chien <[email protected]>

minor clean

77019a4

Signed-off-by: Richard Chien <[email protected]>

support all types

668c66d

Signed-off-by: Richard Chien <[email protected]>

convert between rust type and pb

d1da702

Signed-off-by: Richard Chien <[email protected]>

little reorder

64ee6d0

Signed-off-by: Richard Chien <[email protected]>

move code

ea7c8a6

Signed-off-by: Richard Chien <[email protected]>

pass order data type and order type via message WindowFrame

2ad9939

Signed-off-by: Richard Chien <[email protected]>

limit supported order and offset data types

d3c39cb

Signed-off-by: Richard Chien <[email protected]>

add planner tests

9d2b7c4

Signed-off-by: Richard Chien <[email protected]>

add more planner tests

7cca40e

Signed-off-by: Richard Chien <[email protected]>

align proto message with Rust structures

0aeeadc

Signed-off-by: Richard Chien <[email protected]>

Update src/common/src/types/mod.rs

0e92202

type alias RowsFrameBound to FrameBound<usize> and `RangeFrameBou…

46f45dd

…nd` to `FrameBound<RangeFrameOffset>` Signed-off-by: Richard Chien <[email protected]>

stdrc force-pushed the rc/range-frame-frontend branch from ee5fa37 to 46f45dd Compare February 4, 2024 09:02

stdrc enabled auto-merge February 4, 2024 09:17

stdrc added this pull request to the merge queue Feb 4, 2024

Merged via the queue into main with commit 5092f93 Feb 4, 2024
29 of 30 checks passed

stdrc deleted the rc/range-frame-frontend branch February 4, 2024 09:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(over window): window function `RANGE` frame support in frontend #14648

feat(over window): window function `RANGE` frame support in frontend #14648

stdrc commented Jan 18, 2024 •

edited

Loading

stdrc commented Jan 18, 2024 •

edited

Loading

fuyufjh left a comment

fuyufjh Feb 2, 2024

fuyufjh Feb 2, 2024

chenzl25 Feb 2, 2024

fuyufjh Feb 2, 2024

stdrc Feb 2, 2024 •

edited

Loading

fuyufjh Feb 2, 2024

stdrc Feb 2, 2024

	// Const eval of exprs at the last minute
	plan = const_eval_exprs(plan)?;

feat(over window): window function RANGE frame support in frontend #14648

feat(over window): window function RANGE frame support in frontend #14648

Conversation

stdrc commented Jan 18, 2024 • edited Loading

What's changed and what's your intention?

Checklist

Documentation

Release note

stdrc commented Jan 18, 2024 • edited Loading

fuyufjh left a comment

Choose a reason for hiding this comment

fuyufjh Feb 2, 2024

Choose a reason for hiding this comment

fuyufjh Feb 2, 2024

Choose a reason for hiding this comment

chenzl25 Feb 2, 2024

Choose a reason for hiding this comment

fuyufjh Feb 2, 2024

Choose a reason for hiding this comment

stdrc Feb 2, 2024 • edited Loading

Choose a reason for hiding this comment

fuyufjh Feb 2, 2024

Choose a reason for hiding this comment

stdrc Feb 2, 2024

Choose a reason for hiding this comment

feat(over window): window function `RANGE` frame support in frontend #14648

feat(over window): window function `RANGE` frame support in frontend #14648

stdrc commented Jan 18, 2024 •

edited

Loading

stdrc commented Jan 18, 2024 •

edited

Loading

stdrc Feb 2, 2024 •

edited

Loading